Towards the FAIRification of lab-data

04 July, 2022

Integrated lab solutions

Modern lab requirements

High throughput of samples
Multiple parameters
Many (partly-)automatized techniques
Software and computer systems
Multifaceted and large data streams

A 2020 study of my own: atomic absorption spectroscopy, atomic emission spectroscopy, mass spectrometry, scanning electron microscopy, spectrophotometry, energy-dispersive X-ray spectroscopy, elemental analyser, and high-sensitive balance

Taking command of lab-data

Default commercial software (Cameca, Zeiss, Thermo Fisher)
Prevent tracking data from source to publication
Long-term storage and accessibility throughout the data life-cycle
Monitoring and troubleshooting is reduced to current analysis

Vendor lock-in

Opening-up the black box of lab-data

GUI based dashboards, wizards and dialogs that hide (part) of the transformations and calculations taking place
The reviewer that wants to trace back the origin of data
Old/defunct machinery

Is more data better?

New innovations
Inclusive science
More transparent science (proof of final published values)

xkcd.com

The integrated lab

Data collecting and harmonization
- Parsing of unstructured data
- Data normalization (SQL-like)
Modular processing, analysis, and diagnostics suite
- Count statistics, spectral analysis, regression, …
Online monitoring
- Dashboards of the lab’s long-term reproducibility
- Troubleshooting

The integrated lab

Data collecting and harmonization
- Parsing of unstructured data
- Data normalization (SQL-like)

Modular processing, analysis, and diagnostics suite

Count statistics, spectral analysis, regression, …

Online monitoring

Dashboards of the lab’s long-term reproducibility
Troubleshooting

Data collecting and harmonization

Custom solutions

Deciphering the vendor’s data-model is labor intensive
- Multiple files
- Many observations
- Inconsistent syntax
- Unstructured
Accommodate vendor’s software/data-model updates

An universal solution?

Parsing lab-data

Text data (encoded or decoded)

Unstructured raw data files from analytical laboratory

Provisional solutions

Three possible solutions, which require varying degrees of human intervention:

A mechanism to aid the location of variables based on user input
A human-crafted (and adaptable) rule based system
A natural language processing approach involving self-supervised machine learning

The last two solutions would be preceded by a step entailing text normalization through tokenization.

add 1: requires the input of variable names and their context (i.e., a table or line), whereby regular expression locate the respective variables for subsequent parsing. This approach would thus require considerable knowledge of the end-user considering the raw data and its internal organization. add 2: Tokenization will be performed with cascades of regular expressions for word (entity) delimiters. These delimiters will likely not be based on word boundaries, but instead use a combination of punctuation and tabs as delimiters. On the other hand, special character and alphanumeric combinations, as occur in paths and dates, should constitute one token, and require special consideration. add 3: This method could be reminiscent of part-of-speech tagging in order to recognize the individual entities of the triplet; variables, values, and units. Recognition of larger structures (i.e, tables) might be based on chunking approaches that reminisce the methods serving context free grammar and/or dependency grammar solutions in NLP.

Provisional solutions

Solutions	Human-action	Risk
#1	high	low
#3	medium	medium
#3	low	high

iRODS and user accesibility

Integration with iRODS

Sub-system for automated ingest
Automated workflows
Better collaboration

Accessibility

Interfaces for R and Python (standalone usage)

Evan-Amos, 2011. A SanDisk Cruzer USB drive from 2011, with 4 GB of storage capacity Wikipedia

Implementation

panacea: Portable ANalytical data Aggregation and Coordination for database Entry and Access

C++ for optimal performance with large datasets
R and python bindings for user-friendliness and standalone usage

auxiliary

Updating rirods (irods/irods_client_library_r_cpp) to work with iRODS from R
Restrictive and complex system requirements not ideal for R and C++ integration

Roadmap

Short-term goals

Address the vendor lock-in of lab-data with panacea

New software updates of the vendor-supplied software that impact the output format can be more easily accommodated
Data formats from defunct software and vendors can be more conveniently analysed and/or archived
The integration of (meta)data from different sources can aid online monitoring of lab performance

Long-term goals

The integrated lab will foster:

more efficient labs and innovations
better open science practices
inclusive science

Stimulate a push in the industry of lab equipment towards open sourced software solutions

Thanks

iRODS consortium

Johan Renaudie (MfN Berlin)
Terrell Russell (iRODS)

FAIReLABS

Help us! https://fairelabs.github.io/webpage/